TONAL DisCo: DISSONANCE AND CONSONANCE IN A GAMING ENGINE

Whilst there are several existing toolkits specifically designed for sonification, there has been little investigation into the utilization of computer game engines for sonification. This paper will demonstrate the implementation of a real time game engine for the purpose of sonification and discuss the opportunities and limitations. An important aspect which is lacking in existing sonification toolkits is the ability to sonify streaming data in real-time. Gaming engines not only offer the potential to do this but also offer the ability to visualize data in 3D and in real-time. The sound design of an art exhibition is used as a case study to demonstrate the potential of a computer game editor/sandbox used for visualization and sonification. For the exhibition real-world objects were tracked inside a gallery space and represented in the virtual environment of a computer game, which was displayed on a projector screen. Their movement was sonified into musical form to convey their steady/consistent movement as consonant and their agitated/inconsistent movement as dissonant. Tonal “DisCo” is used to describe their dissonance and consonance rating in both musical tone and visual color. Although the sonification of data into musical structure distorts the accuracy of absolute data values, it does maintain the relationship between data values. This loss of resolution is counteracted by an increase in clarity of data relationships. This case study appropriates the single ratio scale of pitch into both an interval scale of tone and an ordinal scale of octaves in order to express interrelationships.